The technology described herein relates to the processing of video data. More particularly, the technology described herein relates to a method of and apparatus for processing frames of video data.
It is known in the art for video processing systems to generate frames of video data, typically for provision on an electronic display. A video frame is typically represented as a rectangular array of pixels (picture elements) representing an image, where the colour value to be used for each pixel is indicated using an appropriate colour space, e.g. the RGB color space.
Storing the pixel data for a large number of video frames requires a large amount of memory. Accordingly, a number of video encoding methods have been developed to allow the pixel data representing the video frames to be stored in a compressed form.
According to many such video encoding methods, frames in a sequence of video frames are differentially encoded (i.e. in terms of their differences) relative to a so-called “reference frame”, which is a frame for which full resolution pixel data is stored, and which is not defined with reference to another frame. Typically, plural frames of a sequence of video frames are relatively defined with respect to a single reference frame, and this arrangement is repeated over the sequence of video frames.
One such video encoding method uses so-called “motion estimation”, wherein a given frame is divided into plural blocks of, e.g., 16×16 pixels, and each block of pixels is encoded with a vector value (the so-called “motion vector”) pointing to a corresponding block of pixels in the reference frame, and data (the so-called “residual”) describing the differences between the current frame pixel block and the corresponding pixel block in the reference frame. This thereby allows the pixel data for the pixel block of the current frame to be constructed from the pixel data for the pixel block in the reference frame that is pointed to by the motion vector and the residual data describing the differences between that pixel data and the pixel data of the current video frame.
Employing video encoding methods which use reference frames can lead to a significant reduction in memory requirements compared to arrangements where the raw pixel values are stored for each and every frame in a sequence of video frames.
Typically, a new reference frame is generated periodically, e.g. for every N frames in a sequence of frames. The reference frame is then stored in memory for use by the system.
For example, the pixel data for the current reference frame will be generated and stored for use when encoding a sequence of video frames, e.g. to derive the motion vectors and residual values for the frames being encoded in dependence on the reference frame. (Typically, a new reference frame will replace the existing reference frame in memory.)
When decoding a sequence of video frames, e.g. to display a sequence of video frames, the current reference frame will be generated from the encoded video data and, e.g., displayed, and also stored for use when decoding other frames in the sequence that are dependent on the reference frame. (In order to correctly decode differentially encoded video frames, a full resolution reference frame is generated and stored in memory to be used during the decoding process.)
The Applicants believe that there remains scope for improvements to methods of and apparatus for processing frames of video data in a video processing system.